Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding OAuth2 client credentials support for Auth0 connector #2820

Merged
merged 8 commits into from
Mar 21, 2023

Conversation

galvana
Copy link
Contributor

@galvana galvana commented Mar 13, 2023

Closes #2777

Code Changes

  • Changed Auth0's authentication strategy from bearer to oauth2_client_credentials

Steps to Confirm

  • Verified with integration tests and via admin UI

image

image

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Issue Requirements are Met
  • Update CHANGELOG.md

@galvana galvana linked an issue Mar 13, 2023 that may be closed by this pull request
@cypress
Copy link

cypress bot commented Mar 13, 2023

Passing run #788 ↗︎

0 3 0 0 Flakiness 0
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.

Details:

Merge fd259a0 into aafc6c5...
Project: fides Commit: 1a9601291e ℹ️
Status: Passed Duration: 00:36 💡
Started: Mar 14, 2023 12:43 AM Ended: Mar 14, 2023 12:43 AM

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@codecov
Copy link

codecov bot commented Mar 13, 2023

Codecov Report

Patch coverage has no change and project coverage change: +0.16 🎉

Comparison is base (e8e1210) 86.53% compared to head (fd259a0) 86.70%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2820      +/-   ##
==========================================
+ Coverage   86.53%   86.70%   +0.16%     
==========================================
  Files         291      294       +3     
  Lines       16488    16739     +251     
  Branches     2118     2148      +30     
==========================================
+ Hits        14268    14513     +245     
- Misses       1819     1821       +2     
- Partials      401      405       +4     

see 22 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@nicolas-ethyca
Copy link
Contributor

nicolas-ethyca commented Mar 13, 2023

Is there a way to hide the secret?

Copy link
Contributor

@nicolas-ethyca nicolas-ethyca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-----------SAAS REQUEST-----------
PATCH https://dev-tfom-63o.us.auth0.com/api/v2/users/auth0%7C640fa0bc26f817f043e0f445
headers: {'Content-Type': 'application/json', 'Content-Length': '91', 'Authorization': 'Bearer REDACTED'}
body: {
  "email": null, "name": null, "identities": {"0": {"user_id": null}}, "user_id": null
}

response: b'{"statusCode":400,"error":"Bad Request","message":"Payload validation error: \'Expected type string but found type null\' on property email (Email address of this user).","errorCode":"invalid_body"}'
2023-03-13 22:25:12.352 [WARNING] (graph_task:result:118): Retrying erasure_request auth0:users in 1 seconds...
2023-03-13 22:25:13.354 [WARNING] (graph_task:log_end:392): Ending pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7, auth0:users with failure MASKED
2023-03-13 22:25:13.365 [DEBUG] (cache:get_cache:177): Testing Redis connection...
2023-03-13 22:25:13.365 [DEBUG] (cache:get_cache:183): Redis connection succeeded.
2023-03-13 22:25:13.366 [DEBUG] (task_resources:close:195): Closing all task resources for pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7
2023-03-13 22:25:13.383 [ERROR] (logger:_log_exception:22): Client call failed with status code '400'
Traceback (most recent call last):

  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/util/wrappers.py", line 12, in wrap
    loop = asyncio.get_running_loop()
           │       └ <built-in function get_running_loop>
           └ <module 'asyncio' from '/usr/local/lib/python3.10/asyncio/__init__.py'>

RuntimeError: no running event loop


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/usr/local/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
    │    └ <function Thread._bootstrap_inner at 0x7fdb38644ca0>
    └ <WorkerThread(AnyIO worker thread, started 140578673112832)>
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
    │    └ <function WorkerThread.run at 0x7fdb0c4e5900>
    └ <WorkerThread(AnyIO worker thread, started 140578673112832)>
  File "/root/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
             │       │   │      └ ()
             │       │   └ functools.partial(<function approve_privacy_request at 0x7fdb1602b250>, db=<sqlalchemy.orm.session.ExtendedSession object at ...
             │       └ <method 'run' of '_contextvars.Context' objects>
             └ <_contextvars.Context object at 0x7fdb0de78d80>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/api/v1/endpoints/privacy_request_endpoints.py", line 1287, in approve_privacy_request
    return review_privacy_request(
           └ <function review_privacy_request at 0x7fdb16029a20>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/api/v1/endpoints/privacy_request_endpoints.py", line 1139, in review_privacy_request
    process_request_function(privacy_request)
    │                        └ <fides.api.ops.models.privacy_request.PrivacyRequest object at 0x7fdb0de529b0>
    └ <function approve_privacy_request.<locals>._approve_request at 0x7fdb0df9db40>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/api/v1/endpoints/privacy_request_endpoints.py", line 1285, in _approve_request
    queue_privacy_request(privacy_request_id=privacy_request.id)
    │                                        │               └ <sqlalchemy.orm.attributes.InstrumentedAttribute object at 0x7fdb1bde96c0>
    │                                        └ <fides.api.ops.models.privacy_request.PrivacyRequest object at 0x7fdb0de529b0>
    └ <function queue_privacy_request at 0x7fdb1698b6d0>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/service/privacy_request/request_runner_service.py", line 256, in queue_privacy_request
    task = run_privacy_request.delay(
           └ <@task: fides.api.ops.service.privacy_request.request_runner_service.run_privacy_request of fides.api.ops.tasks at 0x7fdb2023...
  File "/root/.local/lib/python3.10/site-packages/celery/app/task.py", line 425, in delay
    return self.apply_async(args, kwargs)
           │    │           │     └ {'privacy_request_id': 'pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7', 'from_webhook_id': None, 'from_step': None}
           │    │           └ ()
           │    └ <function Task.apply_async at 0x7fdb1bd1d3f0>
           └ <@task: fides.api.ops.service.privacy_request.request_runner_service.run_privacy_request of fides.api.ops.tasks at 0x7fdb2023...
  File "/root/.local/lib/python3.10/site-packages/celery/app/task.py", line 572, in apply_async
    return self.apply(args, kwargs, task_id=task_id or uuid(),
           │    │     │     │               │          └ <function uuid at 0x7fdb1bf76440>
           │    │     │     │               └ None
           │    │     │     └ {'privacy_request_id': 'pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7', 'from_webhook_id': None, 'from_step': None}
NoneType: None
           │    │     └ []
           │    └ <function Task.apply at 0x7fdb1bd1d630>
           └ <@task: fides.api.ops.service.privacy_request.request_runner_service.run_privacy_request of fides.api.ops.tasks at 0x7fdb2023...
  File "/root/.local/lib/python3.10/site-packages/celery/app/task.py", line 793, in apply
    ret = tracer(task_id, args, kwargs, request)
          │      │        │     │       └ {'id': '28ea46e0-3c33-4e2d-bdf0-7816def613a2', 'retries': 0, 'is_eager': True, 'logfile': None, 'loglevel': 0, 'hostname': '5...
          │      │        │     └ {'privacy_request_id': 'pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7', 'from_webhook_id': None, 'from_step': None}
          │      │        └ ()
          │      └ '28ea46e0-3c33-4e2d-bdf0-7816def613a2'
          └ <function build_tracer.<locals>.trace_task at 0x7fdb0c3bb910>
  File "/root/.local/lib/python3.10/site-packages/celery/app/trace.py", line 451, in trace_task
    R = retval = fun(*args, **kwargs)
                 │    │       └ {'privacy_request_id': 'pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7', 'from_webhook_id': None, 'from_step': None}
                 │    └ ()
                 └ <bound method run_privacy_request of <@task: fides.api.ops.service.privacy_request.request_runner_service.run_privacy_request...
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/util/wrappers.py", line 15, in wrap
    return asyncio.run(func(*args, **kwargs))
           │       │   │     │       └ {'privacy_request_id': 'pri_0c406c96-6746-43c3-b73b-5c2e5f2986c7', 'from_webhook_id': None, 'from_step': None}
           │       │   │     └ (<@task: fides.api.ops.service.privacy_request.request_runner_service.run_privacy_request of fides.api.ops.tasks at 0x7fdb202...
           │       │   └ <function run_privacy_request at 0x7fdb169f0e50>
           │       └ <function run at 0x7fdb38495d80>
           └ <module 'asyncio' from '/usr/local/lib/python3.10/asyncio/__init__.py'>
  File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
           │    │                  └ <coroutine object run_privacy_request at 0x7fdb0e0b1d20>
           │    └ <method 'run_until_complete' of 'uvloop.loop.Loop' objects>
           └ <uvloop.Loop running=True closed=False debug=False>
> File "/usr/local/lib/python3.10/site-packages/fides/api/ops/service/privacy_request/request_runner_service.py", line 377, in run_privacy_request
    await run_erasure(
          └ <function run_erasure at 0x7fdb1698b250>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/task/graph_task.py", line 809, in run_erasure
    v = delayed(get(dsk, TERMINATOR_ADDRESS, num_workers=1))
        │       │   │    └ __TERMINATE__:__TERMINATE__
        │       │   └ {auth0:users: (<bound method GraphTask.erasure_request of <class 'fides.api.ops.task.graph_task.GraphTask'>:auth0:users>, [{'...
        │       └ <function get at 0x7fdb16ba6dd0>
        └ <function delayed at 0x7fdb16b8d6c0>
  File "/root/.local/lib/python3.10/site-packages/dask/threaded.py", line 89, in get
    results = get_async(
              └ <function get_async at 0x7fdb16b40c10>
  File "/root/.local/lib/python3.10/site-packages/dask/local.py", line 511, in get_async
    raise_exception(exc, tb)
    │               │    └ <traceback object at 0x7fdb0e05e040>
    │               └ ClientUnsuccessfulException("Client call failed with status code '400'")
    └ <function reraise at 0x7fdb16b40af0>
  File "/root/.local/lib/python3.10/site-packages/dask/local.py", line 319, in reraise
    raise exc
          └ ClientUnsuccessfulException("Client call failed with status code '400'")
  File "/root/.local/lib/python3.10/site-packages/dask/local.py", line 224, in execute_task
    result = _execute_task(task, data)
             │             │     └ {__ROOT__:__ROOT__: 0}
             │             └ (<bound method GraphTask.erasure_request of <class 'fides.api.ops.task.graph_task.GraphTask'>:auth0:users>, [{'created_at': '...
             └ <function _execute_task at 0x7fdb16cd1f30>
  File "/root/.local/lib/python3.10/site-packages/dask/core.py", line 119, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
           │      │                │               └ ([{'created_at': '2023-03-13T22:16:28.173Z', 'email': 'nicolas@ethyca.com', 'email_verified': False, 'identities': [{'connect...
           │      │                └ {__ROOT__:__ROOT__: 0}
           │      └ <function _execute_task at 0x7fdb16cd1f30>
           └ <bound method GraphTask.erasure_request of <class 'fides.api.ops.task.graph_task.GraphTask'>:auth0:users>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/task/graph_task.py", line 132, in result
    raise raised_ex  # type: ignore
          └ ClientUnsuccessfulException("Client call failed with status code '400'")
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/task/graph_task.py", line 92, in result
    return func(*args, **kwargs)
           │     │       └ {}
           │     └ (<class 'fides.api.ops.task.graph_task.GraphTask'>:auth0:users, [{'created_at': '2023-03-13T22:16:28.173Z', 'email': 'nicolas...
           └ <function GraphTask.erasure_request at 0x7fdb1698add0>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/task/graph_task.py", line 559, in erasure_request
    output = self.connector.mask_data(
             │    │         └ <function SaaSConnector.mask_data at 0x7fdb1b6120e0>
             │    └ <fides.api.ops.service.connectors.saas_connector.SaaSConnector object at 0x7fdb043dfb80>
             └ <class 'fides.api.ops.task.graph_task.GraphTask'>:auth0:users
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/service/connectors/saas_connector.py", line 418, in mask_data
    client.send(prepared_request, masking_request.ignore_errors)
    │      │    │                 │               └ False
    │      │    │                 └ SaaSRequest(request_override=None, path='/api/v2/users/<user_id>', method='PATCH', headers=[], query_params=[], body='{\n  <m...
    │      │    └ SaaSRequestParams(method='PATCH', path='/api/v2/users/auth0|640fa0bc26f817f043e0f445', headers={'Content-Type': 'application/...
    │      └ <function AuthenticatedClient.send at 0x7fdb1b7fbac0>
    └ <fides.api.ops.service.connectors.saas.authenticated_client.AuthenticatedClient object at 0x7fdb043dc1f0>
  File "/usr/local/lib/python3.10/site-packages/fides/api/ops/service/connectors/saas/authenticated_client.py", line 138, in result
    raise last_exception  # type: ignore
          └ ClientUnsuccessfulException("Client call failed with status code '400'")

fides.api.ops.common_exceptions.ClientUnsuccessfulException: Client call failed with status code '400'
2023-03-13 22:25:13.417 [INFO] (main:log_request:320): Request received | {'method': 'PATCH', 'status_code': 200, 'handler_time': '300.527ms', 'path': '/api/v1/privacy-request/administrate/approve'}

Need to handle cases where null rewrite wont work.

@nicolas-ethyca
Copy link
Contributor

image

@nicolas-ethyca
Copy link
Contributor

Tested and works :)

@galvana galvana merged commit bc955be into main Mar 21, 2023
@galvana galvana deleted the 2777-auth0-client-credentials-flow branch March 21, 2023 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Auth0 Fides Integration missing Client Credentials flow.
3 participants